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Abstract 


The  author  of  this  report  provides  an  overview  of  mental  workload 
theory  and  mental  workload  measurement.  She  also  describes  the 
development,  application,  and  validation  of  the  mental  workload 
modeling  tools  developed  by  the  Human  Research  and  Engineering 
Directorate  of  the  U.S.  Army  Research  Laboratory  (ARL).  These 
ARL  tools,  VACP  (visual,  auditory,  cognitive,  psychomotor) 
option  in  the  improved  performance  research  integration  tool 
(IMPRINT)  and  WinCrew,  can  help  the  designers  of  military 
systems  to  assess  the  mental  workload  associated  with  different 
configurations  of  soldiers  and  equipment  involved  in  the 
performance  of  a  mission.  System  designers  can  conduct  this 
assessment  in  the  concept  development  phase  of  system  design 
and  reduce  the  need  to  build  costly  system  mock-ups. 
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MENTAL  WORKLOAD  AND  ARL  WORKLOAD  MODELING  TOOLS 


1.  Introduction 


The  purpose  of  this  report  is  to  provide  an  overview  of  mental  workload  theory 
and  mental  workload  measurement  and  to  describe  the  development,  application, 
and  validation  of  the  mental  workload  modeling  tools  developed  by  the  Human 
Research  and  Engineering  Directorate  of  the  U.S.  Army  Research  Laboratory 
(ARL).  ARL  has  developed  several  workload  modeling  tools  that  can  help  the 
designer  of  military  systems  to  assess  the  mental  workload  associated  with 
different  configurations  of  soldiers  and  equipment  involved  in  the  performance  of 
a  mission.  Mental  workload  assessment  is  necessary  because  military  systems  are 
becoming  extremely  complex,  requiring  the  system’s  operator  to  process  a  large 
amount  of  information.  The  system  designer’s  task  is  therefore  a  threefold  task. 
The  designer  must  (a)  strive  to  reduce  system  complexity  and  enhance  operator 
performance,  (b)  ensure  that  automation,  if  implemented  in  system  design,  is  used 
where  it  is  most  beneficial  to  the  operator,  and  (c)  ensure  that  the  proposed 
system  does  not  overwhelm  the  operator  with  too  much  information  in  too  short  a 
time.  The  system  designer  can  accomplish  these  tasks  by  assessing  the  mental 
workload  of  the  operator  with  the  ARL  workload  modeling  tools.  Using  these 
tools,  the  system  designer  can  estimate  and  improve  performance  through  the 
study  and  prediction  of  workload.  To  use  these  workload-modeling  tools 
effectively,  however,  the  system  designer  should  understand  the  concept  of 
mental  workload  and  its  importance  to  system  design. 


2.  Mental  Workload  Definitions  and  Concepts 


Mental  workload  is  usually  associated  with  information  processing  tasks,  but  any 
human  activity  includes  mental  processing  and  thus,  mental  workload  (Nachreiner, 
1995).  Driving  a  car,  for  example,  creates  a  certain  level  of  mental  workload  for 
the  driver.  The  driver  must  scan  the  highway,  maintain  control  of  the  vehicle, 
maintain  pressure  on  the  gas  or  brake,  and  be  aware  of  surrounding  vehicles.  All 
these  tasks  require  mental  processing  and  increase  the  mental  workload  of  the 
driver.  The  driver  of  the  car,  as  well  as  the  operator  of  a  military  system,  will 
perform  most  effectively  if  his  or  her  mental  workload  is  optimized  as  he  or  she 
performs  tasks.  Because  the  optimization  of  workload  is  so  critical,  it  has  been 
acknowledged  as  a  goal  for  good  system  design  in  Part  2  of  International 
Organization  for  Standardization  10075,  an  international  standard  for  ergonomic 
principles  (Nachreiner,  1995).  Therefore,  if  a  system  designer  wants  a  good 
system  design,  he  or  she  must  understand  not  only  the  concept  of  workload  but 
also  “optimal”  workload. 
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Hart  (1991)  defines  “optimal”  workload  as  “a  situation  in  which  the  operator  feels 
comfortable,  can  manage  task  demands  intelligently,  and  maintain  good 
performance”  (p.3).  She  uses  an  operational  definition  of  workload,  rather  than  a 
definition  of  the  concept  itself,  because  researchers  do  not  agree  on  one  definition 
of  the  workload  concept  (Huey  &  Wickens,  1993;  Moray,  1988;  Williges  & 
Wierwille,  1979).  Traditionally,  workload  has  been  defined  as  (a)  imposed  task 
demands,  (b)  level  of  performance,  (c)  mental  and  physical  effort  exerted  by  an 
operator,  or  (d)  the  operator’s  perception  (Huey  &  Wickens,  1993).  Most 
contemporary  psychologists  use  definitions  similar  to  the  one  given  by  Hart 
(1991),  which  assumes  that  workload  results  from  some  combination  of  a  specific 
operator  and  the  task  assigned  (Huey 
&  Wickens,  1993). 

Whichever  workload  definition  he  or  she  prefers,  the  system  designer’s  goal  is  to 
optimize  performance  by  optimizing  workload.  The  designer  optimizes  workload 
by  reducing  it  to  just  below  the  level  where  it  has  an  adverse  effect  on 
performance  (Nachreiner,  1995).  This  goal  can  be  difficult  to  achieve,  however, 
because  there  is  sometimes  a  disassociation  between  workload  and  performance. 
The  disassociation  means  that  an  individual’s  mental  workload  may  increase  while 
having  no  effect  on  the  individual’s  performance.  A  number  of  workload  theories 
have  attempted  to  account  for  this  disassociation  and  to  define  the  nature  of  the 
workload  and  performance  relationship.  Understanding  this  workload  literature 
can  help  the  system  designer  to  optimize  workload  in  his  or  her  system  designs 
because  it  helps  the  designer  interpret  the  relationship  between  workload  and 
performance  in  his  or  her  designs. 


3.  Mental  Workload  Theory 


Although  more  than  400  reports  about  the  topic  of  workload  have  been  published 
since  1979,  there  is  no  one  accepted  theory  of  workload,  just  as  there  is  no  one 
accepted  definition  (Huey  &  Wickens,  1993;  Moray,  1988;  Williges  &  Wierwille, 
1979).  Many  of  the  workload  theories  are  based  on  the  information-processing 
model,  which  views  the  human  as  analogous  to  a  computer.  The  cognitive 
energetic  (CE)  theory  (Gaillard  &  Wientjes,  1994)  is  one  example  of  this  type  of 
theory.  CE  theory  assumes  that  each  human  activity  has  its  own  optimal 
energetic  state  at  which  performance  is  most  efficient.  Performance  suffers  when 
the  individual’s  actual  state  deviates  from  the  optimal  state. 

Wickens’  (1991)  multiple  resource  theory  (MRT)  of  attention  is  another 
information-processing  theory  of  workload.  Whereas  CE  theory  explains  a 
performance  decrement  as  a  deviation  from  the  optimal  state,  Wickens’  theory 
views  performance  decrement  as  a  shortage  of  resources.  Wickens’  theory  proposes 
that  the  human  has  a  limited  capacity  for  processing  information.  Thus,  if  an 
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operator  is  asked  to  perform  two  tasks  at  the  same  time,  the  performance  of  one  or 
both  of  the  tasks  may  suffer.  Performance  may  suffer  because  each  task  has  fewer 
available  resources  than  when  each  task  was  performed  separately. 

Inherent  in  both  the  CE  and  MRT  theories  is  the  assumption  that  the  human  is  a 
limited  capacity  processor  who  can  perform  tasks  at  a  finite  rate.  Because  of  this 
assumption,  Moray,  Dessouky,  Adapathya,  and  Kijowski  (1991)  propose  that  the 
appropriate  model  for  studying  workload  is  scheduling  theory.  This  theory  is 
widely  used  in  industrial  engineering.  This  model  views  the  human  as  a  machine 
with  cognitive  tasks  or  “jobs”  to  perform.  A  criterion  is  identified  which  the 
operator  must  meet.  The  rules  of  scheduling  theory  are  then  used  to  decide  in  what 
order  tasks  will  be  done,  how  long  each  job  will  take,  and  whether  the  criterion  can 
be  met. 

The  scheduling  theory  models,  as  well  as  the  CE  and  MRT  models,  assume  that 
people  are  performing  tasks.  Tasks  are  usually  defined  as  behaviors  with  an 
identifiable  beginning  and  end.  Vallacher  and  Wegner  (1987)  have  proposed  a 
workload  theory  that  is  based  on  the  hierarchical  nature  of  tasks.  For  example,  the 
same  act  could  be  described  as  pressing  keys,  typing  a  paper,  proposing  a  new 
research  concept,  and  so  forth.  Their  action-identification  theory  attempts  to 
explain  how  people  identify  tasks  and  when  they  switch  from  one  level  of  task 
processing  to  the  next.  The  theories  just  presented  are  four  of  the  prominent 
theories  or  models  used  to  describe  the  relationship  between  workload  and 
performance.  Note  that  there  is  a  lot  of  overlap  among  these  theories.  Scheduling 
models,  for  example,  are  sometimes  used  to  represent  MRT  and  CE  theory.  This 
overlap  between  workload  theories  is  a  common  problem  in  workload  research. 
Because  subtle  differences  in  workload  theories  can  be  important,  it  is  important 
for  system  designers  to  review  as  many  theories  as  possible.  Further  descriptions 
of  workload  theories  and  models  are  given  in  Huey  and  Wickens  (1993),  Damos 
(1991),  Kramer  (1991),  and  Moray  (1988). 

The  theory  selected  by  ARE  researchers  for  their  workload  modeling  tools  was 
Wickens’  (1991)  MRT.  This  theory  was  selected  for  several  reasons.  It  allows 
the  system  designers  to  predict  when  tasks  can  be  performed  concurrently,  when 
tasks  will  interfere  with  one  another,  and  when  increases  in  the  difficulty  of  one 
task  will  cause  losses  in  performance  of  another  task  (Little  et  al.,  1993).  For 
these  reasons,  MRT  is  the  imderlying  foundation  for  the  prediction  of  workload  in 
the  ARE  modeling  tools.  The  tools  themselves,  however,  are  not  a  theory  but  one 
of  several  methods  for  measuring  and  predicting  workload. 


3.1  Mental  Workload  Measurement 

Just  as  there  are  many  theories  of  workload,  there  are  many  standard  methods  for 
measuring  workload.  Typically,  workload  is  measured  with  either  an  empirical  or 
analytical  approach.  Empirical  techniques  are  used  when  the  system  designer  has 


an  idea  for  improving  an  existing  military  system  or  piece  of  equipment.  The 
system  designer  can  measure  the  effects  of  the  new  design  on  the  system  operator 
by  using  mock-ups  or  simulators  of  the  existing  system.  The  analytical 
techniques  can  be  used  in  these  same  conditions.  However,  the  analytical 
techniques  can  also  be  used  to  predict  workload  when  the  system  is  still  just  a 
concept  and  no  mock-ups  or  simulators  exist. 

All  the  workload  measurement  techniques,  whether  analytical  or  empirical,  vary  in 
sensitivity,  diagnosticity,  and  intrusiveness  (Damos,  1991).  Sensitivity  is  the 
ability  of  a  technique  to  discriminate  between  the  workload  imposed  by  one 
system  or  task  versus  that  imposed  by  another  system  or  task.  Diagnosticity,  on 
the  other  hand,  refers  to  the  ability  of  the  technique  to  discriminate  between 
different  types  of  workload  (e.g.,  visual  versus  cognitive  workload).  Intrusiveness 
refers  to  the  effect,  if  any,  that  the  workload  measurement  technique  has  on  the 
main  task  being  performed  by  the  system  operator.  Because  the  workload 
techniques  vary  in  sensitivity,  diagnosticity,  and  intrusiveness,  a  system  designer 
should  use  a  combination  of  analytical  and  empirical  techniques  when  measuring 
workload.  The  analytical  technique  used  most  often  by  researchers  is  workload 
modeling,  whereas  the  major  empirical  techniques  are  physiological  measures, 
behavioral  measures,  and  subjective  measures. 

Physiological  measures  of  workload  are  reviewed  extensively  by  Kramer  (1991). 
These  measures  of  workload  can  include  measures  of  electroencephalograph 
activity  (EEG),  event-related  potentials  (ERPs),  measures  of  the  magnetic  field 
activity  of  the  brain  (MEG),  measures  of  brain  metabolism  such  as  positron 
emissions  tomography  (PET),  electro-oculograph  (EOG)  activity,  eye  movement, 
and  heart  rate.  Researchers  propose  that  each  of  these  physiological  measures  is 
sensitive  to  some  part  of  workload.  Heart  rate,  for  example,  is  expected  to 
increase  as  workload  increases.  Therefore,  if  the  system  designer  gives  the  system 
operator  a  new  target  detection  system,  the  operator’s  workload  would  be 
expected  to  increase  as  the  number  of  targets  he  or  she  is  required  to  detect 
increases.  Furthermore,  if  heart  rate  is  a  physiological  indicator  of  workload,  then 
the  operator’s  heart  rate  should  increase  as  his  or  her  workload  increases. 
Unfortunately,  physiological  measures  are  not  very  useful  in  early  system  design 
because  at  this  phase,  it  is  highly  unlikely  that  physical  mock-ups  of  the  system 
exist.  Therefore,  the  system  designer  cannot  take  physiological  measures  of  the 
operator’s  workload  in  a  mock-up.  Once  a  prototype  has  been  built,  the 
physiological  measures  are  useful  to  identify  any  areas  where  the  system  user 
might  be  experiencing  excessively  high  levels  of  workload.  The  system  designer 
relying  on  physiological  indices  of  high  workload  needs  to  recognize  that 
individuals  can  sometimes  manage  the  high  workload  they  are  experiencing 
without  its  affecting  their  performance.  In  other  words,  physiological  indices  of 
high  workload  sometimes  do  not  correlate  with  individuals’  performance  (Kramer, 
1991).  However,  the  physiological  measures  can  help  identify  potential  areas  of 
high  workload  that  may  affect  performance.  The  system  designer  can  then  use 
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other  workload  measurement  techniques  to  identify  any  performance  decrements, 
and  the  system  can  be  modified,  if  necessary,  before  it  is  actually  fielded. 

Another  empirical  workload  technique  system  designers  can  use  with 
physiological  techniques  or  to  replace  them  are  behavioral  workload  measures. 
When  system  designers  use  behavioral  measures  as  an  indicator  of  workload  level, 
they  usually  look  at  two  types  of  measures:  a  primary  task  measure  and  a 
secondary  task  measure.  Using  the  primary  task  measure,  the  designer  assesses 
workload  by  examining  some  part  of  the  system  operator’s  ability  to  perform  a 
required  task  such  as  speed  and  accuracy  of  detecting  targets.  For  example,  the 
system  operator  could  be  asked  to  perform  a  primary  task  such  as  detecting  a 
target,  and  the  system  designer  could  measure  the  operator’s  performance  in  terms 
of  speed  and  accuracy  in  that  task.  The  system  designer  then  assumes  that  when 
the  operator’s  workload  increases,  his  or  her  performance  usually  decreases.  This 
performance  decrement  should  be  reflected  in  the  operator’s  performance  by  an 
increase  in  errors  or  longer  target  detection  times.  However,  a  performance 
decrement  does  not  always  occur  because  individuals  can  devise  strategies  that 
allow  them  to  maintain  current  performance  levels  as  workload  increases.  Despite 
this  limitation,  the  primary  task  methodology  is  often  employed  in  field  test 
settings. 

The  secondary  task  methodology  augments  the  primary  task  methodology.  This 
technique  introduces  a  second  task,  which  is  performed  concurrently  with  the 
primary  task.  For  example,  the  operator’s  ability  to  perform  mental  arithmetic 
(secondary  task)  while  detecting  targets  (primary  task)  might  be  assessed.  The 
concept  here  is  that  the  individual  can  use  reserve  processing  capability  to 
perform  the  second  task  (Eggemeier,  Wilson,  Kramer,  &  Damos,  1991).  The 
secondary  task  methodology  is  good  for  laboratory  settings.  It  can  be  used  to 
collect  a  database  of  tasks  that  interfere  with  each  other.  It  is  difficult  to  use  in 
real-world  settings,  however,  because  introduction  of  the  secondary  task  is  so 
invasive.  In  addition,  the  secondary  task  may  be  viewed  as  annoying  or  boring, 
and  the  individual  being  assessed  may  stop  performing  it  (Hart  &  Wickens,  1990). 
To  prevent  these  problems,  system  designers  try  to  have  the  subject  perform  a 
secondary  task,  which  is  a  natural  part  of  the  real-world  setting  (Hart  &  Wickens, 
1990).  For  example,  the  military  system  operator  might  be  asked  to  remember 
critical  radio  call  signs  while  he  or  she  detects  targets. 

The  empirical  technique  that  has  achieved  the  greatest  success  is  subjective 
measurement  of  workload  (Moray,  1988).  Subjective  measurement  usually 
consists  of  asking  the  operator  to  assess  his  or  her  own  workload  level.  In  this 
case,  the  system  designer  assumes  that  when  the  system  operator  expends  more 
mental  energy  to  perform  a  task(s),  the  operator  then  experiences  a  corresponding 
feeling  of  effort  or  exertion  that  can  be  judged  accurately  by  him  or  her.  Most 
researchers  of  mental  workload  agree  that  individuals  find  it  easy  to  provide 
estimates  of  their  workload  and  that  these  estimates  have  good  face  validity 
(Cohen,  Wherry,  &  Glenn,  1996).  However,  the  system  designer  needs  to 
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recognize  that  the  subjects  can  report  that  they  are  experiencing  high  workload 
before  this  high  workload  actually  affects  their  performance.  Therefore,  the 
designer  should  incorporate  performance  measures  along  with  subjective  workload 
ratings  when  he  or  she  is  evaluating  a  system  to  ensure  that  the  workload  reported 
by  the  system  operators  is  actually  affecting  their  performance.  In  addition  to 
this  limitation,  researchers  do  not  agree  about  what  information  should  be  obtained 
from  individuals  or  how  the  information  should  be  used  to  predict  performance 
(Tsang  &  Velazquez,  1996).  This  disagreement  probably  results  from  the  lack  of 
a  workload  theory  that  adequately  predicts  the  relationship  between  workload  and 
performance. 

Despite  the  disagreement  among  researchers,  there  are  a  number  of  different 
methods  such  as  rating  scales,  questionnaires,  or  interviews,  that  system  designers 
can  use  to  collect  subjective  opinions  of  workload.  The  scales  typically  used  to 
obtain  subjective  ratings  of  workload  are  the  subjective  workload  assessment 
technique  (SWAT)  (Reid,  Potter,  &  Dressier,  1989);  the  National  Aeronautics  and 
Space  Administration  (NASA)  task  load  index  (TLX)  (Hart  &  Staveland,  1988); 
and  the  visual,  auditory,  cognitive  and  psychOmotor  (VACP)  model  (McCracken 
&  Aldrich,  1984).  Researchers  have  demonstrated  that  these  scales  do  reflect 
variations  in  workload  demand  for  different  types  of  tasks  (Eggemeier  et  al.,  1991; 
McCracken  &  Aldrich,  1984).  They  are  the  most  sensitive,  most  transferable,  and 
the  least  intrusive  techniques  for  workload  estimation.  Furthermore,  several  scales 
have  demonstrated  global  sensitivity,  which  is  the  ability  to  identify  a  number  of 
different  factors  that  affect  workload.  These  scales,  which  include  SWAT  and 
NASA  TLX,  would  provide  appropriate  workload  indications  if  a  mock-up  of  the 
proposed  system  exists  (Wierwille  &  Eggemeier,  1993).  Lfnfortunately,  the 
requirement  for  a  physical  mock-up  of  the  existing  system  is  a  critical  limitation  of 
the  subjective  measurement  techniques.  The  analytical  techniques,  on  the  other 
hand,  can  be  used  to  predict  mental  workload  when  no  mock-up  exists  and  the 
system  is  just  a  concept. 


4.  ARL  Analysis  Tools 


A  primary  anal5^ical  technique  that  researchers  use  to  measure  and  predict  mental 
workload  is  workload  modeling.  Researchers  have  designed  a  number  of  software- 
based  tools  that  allow  system  designers  to  model  and  predict  mental  workload  and 
its  effect  on  system  performance.  ARL  researchers  developed  two  of  these  tools: 
the  improved  performance  research  integration  tool  (IMPRINT)  and  WinCrew. 
These  tools  have  their  origin  in  two  tools  developed  by  the  Army  in  the  1980s: 
the  manpower-system  evaluation  aid  (MAN-SEVAL)  tool  (Allender,  Kelley, 
Archer,  &  Adkins,  1997)  and  Crewcut  (Little  et  al.,  1993). 
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MAN-SEVAL  was  one  of  the  modules  in  a  suite  of  tools  known  as  Hardware 
Versus  Manpower  III  (HARDMAN  III).  System  designers  used  MAN-SEVAL 
to  identify  the  operator(s)  of  a  proposed  system,  their  tasks,  and  the  operator(s) 
predicted  workload.  Workload  in  MAN-SEVAL  was  based  on  the  mental  aspects 
of  the  VACP  components  of  tasks.  Although  MAN-SEVAL  allowed  designers  to 
predict  mental  workload,  the  software  did  not  allow  them  to  model  the  dynamic 
interactions  between  the  operators’  performance  and  their  mental  workload.  The 
software  simply  provided  the  system  designer  with  two  separate  profiles,  one  for 
workload  and  one  for  system  performance.  Another  tool,  Crewcut  (Little  et  al., 
1993),  took  workload  modeling  a  step  further  and  allowed  system  designers  to 
model  the  dynamic  relationship  between  mental  workload  and  performance. 

System  designers  could  use  Crewcut  to  compare  the  operator  workload  imposed 
by  various  crew  station  designs.  More  importantly,  Crewcut  also  allowed  system 
designers  to  develop  models  of  how  crew  members  manage  workload.  The 
designers  modeled  the  crew  members’  workload  management  using  a  default  set  of 
basic  workload  management  strategies  available  in  the  software  or  workload 
management  strategies  they  defined  from  their  own  research.  These  strategies 
helped  the  designers  to  obtain  an  accurate  picture  of  the  relationship  between 
workload  and  performance.  For  example,  in  a  real-world  situation,  a  crew  member 
may  have  three  tasks  that  need  to  be  performed  at  the  same  time.  Realizing  that 
he  or  she  cannot  maintain  effective  performance,  the  crew  member  gives  one  of  the 
tasks  to  another,  less  busy  crew  member.  Because  reallocating  a  task  to  another 
qualified  crew  member  is  one  of  the  default  strategies  in  Crewcut,  the  system 
designer  could  model  this  situation  and  obtain  an  accurate  estimate  of 
performance.  At  the  time  Crewcut  was  developed,  this  unique  feature  was  not 
available  in  any  other  workload  modeling  tool. 

In  1992,  the  organizations  that  developed  Crewcut  and  MAN-SEVAL  merged  into 
one  organization,  the  U.S.  Army  Research  Laboratory.  Because  system  designers 
needed  both  the  fast  turn-around  analysis  offered  by  MAN-SEVAL  and  the  higher 
fidelity  analysis  offered  by  Crewcut,  a  decision  was  made  to  maintain  both 
modeling  capabilities  as  Microsoft  Windows™ -based  software  tools.  MAN- 
SEVAL  was  incorporated  into  a  larger  software  package  called  IMPRINT 
(Allender  et  al.,  1997)  as  the  “VACP”  option.  Because  IMPRINT  is  the  principal 
human  performance  modeling  tool  for  ARL  and  the  U.S.  Army,  WinCrew 
workload  modeling  capabilities  were  incorporated  into  IMPRINT  as  the  advanced 
workload  analysis.  WinCrew  is  also  maintained  as  a  stand-alone  commercially 
available  tool. 

The  system  designer  uses  IMPRINT  to  estimate  the  likely  performance  of  a  new 
military  system  by  building  models  of  each  operational  or  maintenance  mission 
that  the  system  will  be  required  to  accomplish  ^icro Analysis  and  Design,  1998). 
Because  one  of  the  critical  components  of  operator  and  system  performance  is 
operator  workload,  IMPRINT  has  two  options  for  generating  workload  profiles. 
The  simpler  workload  option  is  the  VACP  option.  The  system  designer  would 
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use  this  option  early  in  the  design  process  when  many  of  the  details  of  the  crew 
stations  may  not  yet  be  identified.  The  other  option,  advanced  workload 
analysis,  is  the  same  as  WinCrew.  This  option  gives  the  system  designer  the 
capability  to  perform  a  more  detailed  and  dynamic  workload  analysis  than  the 
VACP  option.  However,  because  the  advanced  workload  option  associates 
operator  workload  with  the  equipment  interfaces  the  crew  member  uses,  the 
system  design  must  be  at  a  more  advanced  stage  than  with  the  VACP  option. 
Therefore,  the  system  designer  would  usually  begin  a  system  design  analysis  using 
the  VACP  method  and,  if  necessary,  would  apply  the  advanced  workload  method 
as  the  analysis  matures  (Archer,  1998).  The  imderlying  structure  for  both  the 
VACP  option  and  the  advanced  workload  analysis  option,  as  well  as  IMPRINT 
itself,  is  task-network  modeling  and  discrete  event  simulation. 

Task  network  models  are  computer-based  simulation  models  that  the  system 
designer  can  use  to  predict  task  and  procedure  execution  and  mental  workload. 
The  model  consists  of  the  tasks  needed  to  accomplish  a  particular  job  or  mission, 
the  amount  of  time  it  takes  each  task  to  execute,  the  sequence  by  which  the  tasks 
are  performed,  and  the  individual  or  operator  who  performs  each  task. 

To  build  a  task-network  model,  the  system  designer  begins  with  a  mission  or  job 
that  an  operator(s)  will  be  performing.  Next,  the  system  designer  reduces  the 
mission  to  the  specific  tasks  and  subtasks  that  the  operator(s)  need  to  accomplish 
to  perform  the  mission.  He  or  she  then  enters  the  sequence  by  which  the 
operator(s)  perform  the  tasks.  This  process  is  relatively  easy  for  the  designer  to 
perform  and  can  be  done  early  in  the  design  process.  It  has  the  added  benefit  of 
forcing  the  designer  to  think  in  great  detail  about  the  entire  proposed  system.  In 
addition  to  the  tasks  to  be  performed  and  their  sequence,  the  system  designer 
building  a  task-network  model  must  also  estimate  the  time  required  for  the 
operator  to  complete  each  task.  These  task  times  are  then  entered  into  the 
computer  model  and  are  used  to  calculate  overall  system  performance  time.  If  the 
task  network  model  is  a  deterministic  model  with  non-parallel  tasks,  then  the 
times  entered  for  each  task  will  equal  the  estimated  mission  time.  With  a 
stochastic  task-network  model,  however,  the  times  for  each  task  can  be  drawn 
from  a  distribution  of  times.  The  resulting  system  performance  times  for  the 
stochastic  model  are  an  average  of  the  randomly  selected  times.  Because  human 
behavior  varies,  the  stochastic  task-network  model  is  a  more  realistic 
representation  of  system  operators  than  the  deterministic  model.  Therefore,  the 
IMPRINT  software  is  based  on  a  stochastic  rather  than  on  a  deterministic  task 
network  modeling  technique. 

In  addition  to  being  a  stochastic  task-network  modeling  tool,  IMPRINT  uses 
discrete  event  simulation  to  model  human  performance.  Discrete  simulation  and 
continuous  simulation  are  two  broad  classes  of  simulation  models.  The  type  of 
simulation  an  analyst  uses  depends  upon  the  types  of  problems  to  be  solved. 
“For  example,  a  model  of  traffic  flow  on  a  freeway  would  be  discrete  if  the 
characteristics  of  and  movement  of  individual  cars  are  important.  Alternatively,  if 


8 


the  cars  can  be  treated  ‘in  the  aggregate,’  the  flow  of  traffic  can  be  described  by 
differential  equations  in  a  continuous  model”  (Law  &  Kelton,  1991).  Because  the 
system  designer  is  intei'ested  in  the  operator’s  tasks  individually,  not  as  an 
aggregate,  discrete  simulation  was  selected  as  the  basis  for  IMPRINT.  More 
specifically,  IMPRINT  is  based  on  a  discrete  event  simulation.  A  discrete  event 
simulation  is  appropriate  when  the  elements  of  the  process  modeled  have  a 
distinct  beginning  and  ending  as  do  all  the  missions  and  tasks  modeled  with 
IMPRINT.  The  specific  task  network  tool  and  discrete  event  simulation  model 
used  in  the  IMPRINT  software  is  the  commercially  available  MicroSaint™. 

MicroSaint™  is  a  task  network  simulation  language  developed  by  MicroAnalysis 
and  Design,  Inc.,  for  the  U.S.  Army  Medical  Research  and  Development 
Command.  MicroSaint™  makes  it  simple  for  system  designers  to  build  task 
network  models  by  providing  a  graphical  interface  and  flow  chart  approach  to 
modeling.  Using  MicroSaint™,  the  system  designer  builds  a  graphical 
representation  of  the  task  network  that  is  also  a  graphical  representation  of  the  job 
or  mission  the  system  operator  will  be  performing.  Figure  1  depicts  an  example  of 
a  graphical  representation  of  a  task  network.  As  Figure  1  depicts,  the  system 
designer  uses  rounded  rectangles  to  represent  modeled  tasks  and  arrows  to  connect 
the  tasks  to  represent  the  order  of  task  execution  within  the  mission.  Once  the 
designer  has  completed  the  task  network,  the  MicroSaint™  software  links  the 
tasks,  task  times,  and  individual  performance  together  in  the  model  and  simulates 
the  system  operator  performing  a  mission.  The  system  designer  uses  a  modified 
version  of  the  MicroSaint™  graphical  interface  to  model  the  operator’s  tasks  with 
IMPRINT.  In  addition  to  modeling  the  tasks  and  task  times,  however,  the  system 
designer  can  also  use  IMPRINT  to  model  operator  mental  workload  with  either  of 
its  two  options,  VACP  or  advanced  workload  analysis. 


5.  VACP  Method 


VACP  is  based  on  a  tailored  version  of  the  task  analysis/workload  (TAWL) 
methodology  (Hamilton,  Bierbaum,  8c  Fulford,  1991)  which  evolved  from  the 
Army  light  helicopter  experimental  (LHX)  program.  The  first  step  in  a  TAWL  is 
to  identify  the  tasks  that  are  necessary  to  operate  a  proposed  system.  Similarly, 
the  system  designer  building  a  VACP  IMPRINT  model  must  first  identify  the 
tasks  required  for  operating  the  proposed  system.  However,  with  IMPRINT,  the 
designers  do  not  have  to  put  all  the  tasks  for  the  proposed  system  into  one  model. 
If  necessary,  each  mission  performed  by  the  system  and  its  tasks  can  be  a 
separate  model.  The  tasks  incorporated  into  each  model  generally  should  be  tasks 
that  are  performed  in  seconds,  minutes,  or  hours  rather  than  tasks  that  take  days 
or  weeks.  For  example,  system  designers  modeled  a  howitzer  crew,  which 
included  emplacing  the  howitzer,  conducting  a  fire  mission,  and  displacing  the 
howitzer  as  tliree  separate  missions  rather  than  as  one  lai’ge  mission. 
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After  the  system  designers  have  identified  the  tasks  necessary  to  complete  the 
mission,  they  must  identify  the  operators  of  the  system.  Furthermore,  because 
IMPRINT  models  are  models  of  individuals  moving  from  task  to  task,  the 
designer  must  also  designate  which  operator  is  performing  which  tasks.  Once  the 
tasks  and  operators  have  been  identified,  the  system  designer  must  specify  tie 
order  in  which  the  tasks  will  be  performed  and  the  times  to  perform  the  tasks. 
The  system  designer  usually  gets  this  information  from  a  subject  matter  expert 
(SME).  The  SME  for  an  existing  system  would  be  the  system  operator.  If  the 
system  is  still  in  the  concept  phase,  then  the  SME  could  be  someone  who  is 
operating  an  existing  system  that  is  similar  to  the  proposed  system.  The  SME 
would  also  help  the  system  designer  to  estimate  the  workload  values  for  each  task, 
which  is  the  next  step  in  the  VACP  analysis.  The  system  designer  estimates 
workload  for  each  task,  based  upon  the  mental  resources  the  operator  needs  to 
perform  each  task.  With  the  VACP  method,  the  resources  the  operator  is 
expected  to  use  for  any  task  may  be  visual,  auditory,  cognitive,  psycho  motor,  or  a 
combination  of  any  of  these  four  resources.  After  the  system  designer  selects  the 
resources  the  operators  use  for  each  task,  he  or  she  then  enters  numerical  values 
for  the  selected  resources  using  scales  developed  by  McCracken  and  Aldiich 
(1984)  and  enhanced  by  Bierbaum,  S^abo,  and  Aldrich  (1989). 
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The  scale  for  each  resource  is  based  on  a  7-point  interval  and  contains  verbal 
anchors  that  describe  the  behaviors  expected  for  each  interval.  As  an  example  of 
the  scales  that  the  system  designer  is  using  to  rate  workload,  the  scale  and  verbal 
anchors  for  the  visual  resource  are  presented  in  Table  1 .  The  system  designer 
selects  values  from  the  scales  for  each  resource  being  used  for  each  task  and  enters 
the  values  into  the  IMPRINT  software. 

Table  1 

Visual  Workload  Scale  for  the  VACP  Option 


Scale  value  Visual  scale  descriptor 


0.0  No  visual  activity 

1 .0  Visually  register  or  detect  (detect  occurrence  of  image) 

3.7  Visually  discriminate  (detect  visual  differences) 

4.0  Visually  inspect  or  check  (discrete  inspection  or  static  condition) 

5.0  Visually  locate  or  align  (selective  orientation) 

5.4  Visually  track  or  follow  (maintain  orientation) 

5.9  Visually  read  (symbol) 

7.0  Visually  scan,  search,  or  monitor  (continuous  or  serial  inspection, 

multiple  conditions) 


After  the  system  designer  enters  the  workload  ratings  for  each  resource  required 
by  each  task  performed  by  each  operator,  the  IMPRINT  software  sums  the 
workload  ratings  within  each  resource  across  concurrent  tasks.  Although  the 
values  for  each  resource  are  limited  to  a  7-point  scale,  an  operator’s  workload  is 
often  higher  than  seven.  An  operator’s  workload  can  exceed  seven  because  the 
software  is  adding  the  workload  for  all  tasks  that  are  occurring  simultaneously. 
For  example,  the  same  operator  could  perform  two  tasks  at  once.  The  first  task 
has  a  visual  rating  of  4.1  and  the  second  one  has  a  visual  workload  rating  of  3.2. 
For  the  amount  of  time  these  two  tasks  overlap,  the  operator’s  total  visual  rating 
would  be  7.3  (MicroAnalysis  and  Design,  1998).  The  software  computes  the 
workload  rating  while  the  simulation  is  running.  The  simulation  run  provides  the 
system  designer  with  graphs  and  tables  of  the  workload  values  over  time  for  each 
operator  of  die  system.  The  system  designer  can  examine  the  workload  graph  and 
determine  where  the  workload  peaks  are  and  which  tasks  were' being  performed  at 
that  time  and  contributed  to  the  peaks.  If  the  workload  graph  shows  that  all 
operators  had  peak  workload  at  the  same  time,  then  the  system  designer  should 
consider  automating  some  of  the  tasks  the  operators  are  performing  during  that 
time  period.  If  the  tasks  cannot  be  automated,  then  the  system  designer  might 
reduce  the  operators’  workload  by  redesigning  some  of  these  tasks  and  making 
them  less  resource  intensive.  If,  on  the  other  hand,  the  workload  graph  indicates 
that  one  operator  has  a  high  workload  peak  and  another  operator  has  a  low 
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workload,  then  the  system  designer  might  decide  to  reallocate  some  of  the  tasks  to 
the  less  busy  operator.  Thus,  the  workload  graph  allows  the  system  designer  to 
decide  when  tasks  should  be  redesigned,  automated,  or  allocated  to  another 
operator  in  order  to  reduce  the  workload  of  the  operator. 

Although  the  VACP  method  provides  the  system  designer  with  workload 
estimates,  the  workload  for  an  operator  does  not  affect  the  operator’s 
performance  during  a  simulation  run.  On  the  other  hand,  the  advanced  workload 
analysis  option  allows  the  system  designer  to  model  the  dynamic  relationship 
between  workload  and  performance. 


6.  Advanced  Workload  Analysis 


The  relationship  between  workload  and  performance  is  complicated.  It  is  not 
simply  that  as  workload  increases  performance  decreases.  Instead,  the 
relationship  between  workload  and  performance  is  traditionally  described  as  an 
inverted  “U”  because  decrements  in  performance  may  occur  if  workload  is  either 
too  low  or  too  high.  Furthermore,  there  can  be  a  disassociation  between  workload 
and  performance  at  certain  levels.  This  means  that  as  workload  increases,  the 
operator’s  performance  may  not  decrease  because  the  operator  has  a  strategy  for 
bandling  task  demands  to  compensate  for  the  increased  workload.  Hart  (1989) 
proposed  that  operator  workload  strategies  play  an  important  role  “in  determining 
the  relationship  between  objective  task  demands,  experienced  workload,  and 
system  performance”  (p.4).  The  advanced  workload  analysis  feature  of 
IMPRINT  allows  the  system  designer  to  incorporate  operator  workload 
management  strategies  into  the  workload  model.  Thus,  the  advanced  workload 
analysis  feature  provides  the  system  designer  with  a  more  detailed  representation 
of  the  relationship  between  workload  and  system  performance  than  the  VACP 
method  does. 

The  procedure  for  building  a  model  with  the  advanced  workload  analysis  feature  is 
very  similar  to  the  procedure  used  with  the  VACP  method.  To  obtain  a  workload 
estimate  using  the  advanced  workload  analysis  option,  the  system  designer  first 
selects  a  job  or  mission  that  the  operator(s)  of  the  proposed  system  will  perform. 
Next,  the  system  designer  decomposes  the  selected  mission  into  the  tasks  the 
operator(s)  will  be  performing  in  order  to  accomplish  this  mission.  The  task 
sequence  and  the  operator(s)  who  will  perform  these  tasks  are  then  identified. 
The  procedure  at  this  point  has  been  identical  to  VACP.  However,  unlike  the 
VACP  method,  the  advanced  workload  analysis  option  links  workload  to  the 
specific  equipment  used  by  the  operator.  It  does  this  because  it  contains  ^ 
embedded  workload  calculation  algorithm.  This  algorithm,  which  is  depicted  in 
Figure  2,  is  based  on  a  variation  of  the  workload  index  model  (North  &  Riley, 
1989).  It  calculates  workload,  based  on  the  resources  being  used  by  the  operator. 
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and  incorporates  the  fact  that  multiple  tasks  are  being  performed  simultaneously. 
In  addition,  the  algorithm  relates  the  resources  used  to  crew  station  displays  and 
control  surfaces  (Little  et  ah,  1993).  Because  the  advanced  workload  analysis 
option  contains  this  algorithm,  the  system  designers  must  specify  the  equipment 
interfaces  (e.g.,  keyboard,  helmet-mounted  display)  that  operators  will  be  using  to 
accomplish  each  of  their  assigned  tasks.  Furthermore,  the  designer  must  also 
specify  the  mental  resources  a  crew  member  uses  with  each  equipment  interface  as 
he  or  she  performs  each  task. 
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Wt  =  instantaneous  workload  at  time  T 
i,  j  =  1 . .  .1  are  the  interface  channels 

nx,i  =  number  of  tasks  occurring  at  time  T  with  nonzero  attention  to  channel  i 

t  =  1 . .  .m  are  the  operator's  tasks  or  activities 

a  u  =  attention  to  channel  i  required  to  perform  task  t 

Ci,j  =  conflict  between  channels  i  and  j 

Ci,i=  conflict  within  channel  I 

1 .  if  at  a,,i  or  asj  =  0,  then  (at,i  +  asj  =0), 

2.  if  ay  or  as.F  0,  then  (ay  +  a*,!  =  0), 

3.  ifnij  is <or=  1, Cu=0. 


Figure  2.  Workload  algorithm. 


The  VACP  method  limits  the  system  designer  to  four  mental  resources:  visual, 
auditory,  cognitive,  and  psychomotor.  With  the  advanced  workload  method, 
these  resources  are  expanded  to  a  set  of  five  resources:  visual,  auditory,  cognitive, 
psychomotor,  and  speech.  In  addition,  the  system  designers  can  create  their  own 
resources.  For  example,  the  system  designers’  research  may  indicate  that  the 
tactile  resource  is  important  for  their  design.  They  can  then  add  this  to  the 
resource  list  in  the  advanced  workload  analysis  option.  However,  no  default 
scales  are  available  to  help  the  designers  estimate  workload  for  this  resource. 
They  must  substantiate  values  for  this  resource,  based  on  current  research.  If  the 
system  designers  choose  to  use  the  default  resources,  they  can  rate  the  amount  of 
each  of  these  resources  required  to  do  a  task  using  7-point  rating  scales  very 
similar  to  the  McCracken  and  Aldrich  (1984)  scales  used  with  the  VACP  method. 
However,  the  McCracken  and  Aldrich  scales  have  been  revised  in  the  advanced 
workload  analysis  option.  The  original  scales  were  developed  for  estimating 
workload  for  the  Army’s  light  helicopter.  When  the  (former)  Human  Engineering 
Laboratory  was  developing  the  Crewcut  tool,  SMEs  with  armored  vehicle  and 
psychology  expertise  studied  the  helicopter  scales.  Based  on  the  SMEs 
recommendations,  some  of  the  scale  values  were  revised.  In  addition,  the 
psycho  mo  tor  resource  was  divided  into  two  separate  resources:  motor  and 
speech.  Because  the  advanced  workload  analysis  option  is  identical  to  WinCrew 
and  WinCrew  is  the  Windows'^"  version  of  Crewcut,  the  revised  scales  are  used  in 
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the  advanced  workload  option  to  estimate  workload  for  each  resource.  The 
revised  scales  are  provided  in  Table  2.  Just  as  with  the  VACP  method,  the 
system  designers  must  use  these  scales  to  estimate  the  resources  required  for  each 
task  an  operator  performs.  Although  the  system  designers  are  using  similar  scales 
to  rate  workload  with  both  the  VACP  and  advanced  workload  analysis  options, 
the  method  the  software  uses  to  calculate  the  workload,  based  on  the  ratings,  is 
very  different.  Both  of  the  calculation  methods  are  based  on  MRT,  but  the 
implementation  of  this  theory  in  the  advanced  workload  analysis  option  is  much 
more  sophisticated. 

According  to  MRT,  when  an  individual  performs  a  task,  he  or  she  requires 
different  mental  operations  and  to  some  extent,  each  operation  uses  the  mental 
processing  resources  necessary  to  accomplish  the  task.  These  mental  resources 
are  limited,  and  a  supply-and-demand  problem  occurs  when  the  individual 
performs  two  or  more  tasks  that  require  a  single  resource.  As  a  result  of  the  time 
sharing  of  resources,  some  task  performance  times  may  increase,  the  probability 
of  successfully  completing  a  task  may  change,  or  performance  times  may  decrease 
(Little  et  al.,  1993).  These  MRT  concepts  are  the  vmderlying  assumptions  for 
both  workload  options  in  IMPRINT.  MRT,  however,  also  explains  how  two 
tasks  can  conflict  with  each  other. 

According  to  the  multiple  resource  model,  two  concurrent  tasks  will  suffer  greater 
interference  to  the  extent  that  the  component  tasks  are  more  difficult  (demand 
more  resources)  and  that  the  components  compete  for  overlapping  resources. 
Fmthermore,  the  effects  of  difficulty  and  resource  overlap  interact.  The  greater 
the  degree  of  resource  overlap,  the  more  pronounced  will  be  the  effect  of  the  level 
of  difficulty  of  one  task  on  the  level  of  performance  of  another  task  (Little  et  al., 
1993,  p  9). 

In  the  VACP  option,  conflict  between  tasks  is  not  considered  in  the  calculation  of 
workload.  The  workload  calculation  is  simply  the  sum  of  the  workload  ratings 
within  each  resource  across  concurrent  tasks.  The  workload  algorithm  in  the 
advanced  workload  analysis  option,  however,  does  incorporate  the  MRT  findings. 
It  sums  the  resource  demands  and  includes  penalties  for  situations  when  two  tasks 
require  the  same  resources  and  for  situations  when  the  use  of  different  resotirces 
causes  interference.  The  workload  algorithm  itself  is  presented  in  Figure  2. 

The  first  part  of  the  workload  algorithm  computes  the  resource  demands  for  all 
active  tasks.  Therefore,  each  time  a  new  task  is  started,  the  algorithm  adds  the 
workload  ratings  for  each  resomce  for  the  new  task  and  all  other  tasks  being 
performed  at  that  time.  The  next  two  terms  within  the  second  bracket  of  the 
equation  compute  the  penalties  for  two  tasks  using  the  same  resource  at  the  same 
time  and  two  tasks  requiring  different  resources  at  the  same  time.  For  example,  if 
one  task  requires  a  system  operator  to  look  at  a  computer  screen  on  the  right  side, 
while  a  second  task  simultaneously  requires  the  operator  to  look  at  a  computer 
screen  on  the  left  side,  then  the  equation  assigns  a  penalty  to  the  task.  In  this  case. 
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Table  2 


Revised  UH-60  Helicopter  Workload  Component  Scales 


Scale  New 

value  Descriptors  values 


Visually  unaided  (naked  eye) 

1 .0  Visually  register  or  detect  (detect  occurrence  of  image) 

3.7  Visually  discriminate  (detect  visual  differences) 

4.0  Visually  inspect  or  check  (discrete  inspection  or  static  condition) 
5.0  Visually  locate  or  align  (selective  orientation) 

5.4  Visually  track  or  follow  (maintain  orientation) 

5.9  Visually  read  (symbol) 

7.0  Visually  scan  or  search  monitor  (continuous  or  serial  inspection, 
multiple  conditions) 


3.0 

5.0 

3.0 

4.0 

4.4 

5.0 

6.0 


Visually  aided  (with  NVGs) 

4.0  Visually  register  or  detect  (detect  occurrence  of  image)  with  NVGs  5.0 

4.8  Visually  inspect  or  check  (discrete  inspection  or  static  condition  (with  NV Gs)  5.0 

5.0  Visually  discriminate  (detect  visual  differences)  with  NVGs  7 .0 

5.6  Visually  locate  or  align  (selective  orientation)  with  NVGs  5.0 

6.4  Visually  track  or  follow  (maintain  orientation)  with  NVGs  5.4 

7.0  Visually  scan,  search,  or  monitor  (continuous  or  serial  multiple  conditions)  7.0 

with  NVGs 


Auditory 

1 .0  Detect  or  register  sound  (detect  occurrence  of  sound) 

2.0  Orient  to  sound  (general  orientation  or  attention) 

4.2  Orient  to  sound  (selective  orientation  or  attention) 

4.3  Verify  auditory  feedback  (detect  occurrence  of  anticipated  sound) 

4.9  Interpret  semantic  content  (speech)  simple 

(1  to  2  words)  complex  sentences 

6.6  Discriminate  sound  characteristics  (detect  auditory  difference) 
7.0  Interpret  sound  patterns  (pulse  rates,  etc.) 

Cognitive 

1.0  Automatic  (simple  association) 

1 .2  Alternative  selection 

3.7  Sign  or  Signal  recognition 

4.6  Evaluation  or  judgment  (consider  single  aspect) 

5.3  Encoding  or  decoding,  recall 

6.8  Evaluation  or  judgment 

7.0  Estimation,  calculation,  conversion 

Rehearsal 


1.0 

2.0 

4.2 

4.3 
3.0 
6.0 
6.6 
7.0 


1.0 

1.2 

3.7 
4.6 
5.3 

6.8 
6.8 
5.0 


Psychomotor  (this  scale  was  divided  into  speech  and  motor  in  revised  scale 

Speech 

1 .0  Speech  simple  (1  to  2  words) 

Complex  (sentence) 

Motor 

2.2  Discrete  actuation  (button,  toggle,  trigger) 

2.6  Continuous  adjustlve  (flight  control,  sensor  control) 

4.6  Manipulative 

5.8  Discrete  adjustment  (rotary,  vertical  thumb  wheel,  lever  position) 

6.5  Symbolic  production  (writing) 

7.0  Serial  discrete  manipulation  (keyboard  entries) 


2.0 

4.0 


2.2 

2.6 

4.6 

5.5 

6.5 
7.0 
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the  penalty  would  be  that  one  task  could  not  be  performed.  In  other  cases,  the 
penalty  might  be  that  a  task’s  time  is  increased.  The  penalty  is  assigned  because 
the  two  tasks  are  using  the  same  resource  at  the  same  time.  The  system  designer 
determines  the  amount  of  interference  between  resources  being  used  by  concmxent 
tasks  by  entering  values  into  a  conflict  matrix  provided  in  the  software.  This 
conflict  matrix  displays  each  resource  paired  with  the  equipment  interface  that 
uses  the  resource.  For  each  resource  and  interface  pair,  the  designers  enter  conflict 
values,  based  on  guidance  from  their  own  research,  or  the  software  can  provide 
default  values,  based  on  the  MRT  literature.  The  conflict  values,  which  can  range 
from  0  (represents  no  conflict)  to  1.0  (represents  total  conflict),  will  be  unique  to 
each  system  design  because  the  values  are  linked  to  both  equipment  interfaces  and 
resources.  Each  design  will  have  a  different  set  of  equipment  interfaces  that  use 
specific  resources  and  therefore  its  own  set  of  conflict  values. 

After  the  system  designers  have  provided  conflict  values  and  workload  ratings  for 
each  operator  for  each  task,  the  algorithm  calculates  the  workload  for  each 
operator  before  a  new  task  begins,  at  the  start  of  a  new  task,  and  after  a  task  is 
completed.  In  addition  to  calculating  this  overall  workload,  the  advanced 
workload  analysis  option  allows  system  designers  to  specify  how  the  system 
operator  will  manage  the  workload.  They  do  this  with  the  workload  management 
strategies. 


7.  Workload  Management  Strategies 


Workload  management  strategies  allow  the  system  designer  to  account  for  the  fact 
that  individuals  employ  different  strategies  for  performing  tasks  during  conditions 
of  work  overload.  The  need  to  model  workload  management  strategies  was 
initially  suggested  by  Hart  (1989)  who  argued  that  individuals  learn  and  adapt  and 
therefore  respond  dynamically  to  changes  in  their  mental  workload.  She  explained 
that  individuals  learn  strategies  for  adapting  to  increasing  workload  demands. 
When  workload  becomes  too  high,  the  individual’s  strategy  may  deteriorate  and  an 
observer  could  then  see  a  change  in  the  individual’s  performance  as  a  result  in  this 
deterioration  in  strategies.  It  is  important  to  have  these  strategies  in  the  advanced 
workload  analysis  option  because  they  allow  the  system  designer  to  more 
accurately  estimate  the  effects  of  workload  on  performance. 

To  use  the  workload  management  strategies,  the  system  designer  must  first  enter  a 
workload  threshold.  It  must  be  a  number  >  0.  This  threshold  represents  the  point 
where  the  proposed  system’s  operator  will  be  considered  overloaded  and  a 
workload  management  strategy  will  need  to  be  applied.  For  example,  if  the 
system  designer  selects  40  as  the  threshold  number  for  a  particular  operator,  then 
any  time  that  operator’s  workload  exceeds  40,  he  or  she  will  be  considered 
overloaded.  At  this  time,  the  overloaded  operator  will  employ  the  workload 
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management  strategy  selected  by  the  system  designer  to  handle  the  overload.  An 
operator  is  also  considered  to  be  overloaded  any  time  multiple  tasks  require  a 
resource  interface  pair  that  has  been  assigned  a  conflict  value  of  1.0.  When  this 
situation  occurs,  the  operator  will  use  the  workload  management  selected  by  the 
system  designer  to  handle  the  overload. 

The  advanced  workload  analysis  option  includes  six  workload  management 
strategies  for  overload  conditions: 

1 .  No  effect.  All  tasks  are  performed,  regardless  of  the  workload  value. 
This  is  the  default  strategy. 

2.  Do  not  begin  the  next  task.  The  next  task  is  not  started  by  any  other 
operator.  This  is  sometimes  referred  to  ask  “task  shedding.” 

3.  Perform  tasks  sequentially,  beginning  with  ongoing  tasks  and  then 
performing  the  next  task. 

4.  Interrupt  ongoing  tasks  in  favor  of  starting  the  next  task.  Restart 
ongoing  tasks  in  “windows  of  opportunity”  (i.e.,  when  the  operator’s  workload 
drops  below  the  threshold  value). 

5.  Reallocate  the  next  task  to  the  contingency  operator  if  a  contingency 
operator  has  been  designated.  (The  contingency  operator  can  be  an  automated 
system.) 

6.  Reallocate  ongoing  tasks  to  the  contingency  operator  or  automation. 

In  addition  to  these  default  strategies,  system  designers  can  create  their  own 
strategies  for  the  operators  to  use.  They  create  these  strategies  by  combining  pre¬ 
defined  system  variables  and  arithmetic  and  logical  operators  into  “if-then-else” 
statements.  Variables  the  designers  can  use  in  the  expressions  are 

P  -  Priority  of  the  next  task  (each  task  can  be  assigned  a  priority  from  1  to  5) 

H  -  Highest  priority  of  any  ongoing  task 

T  -  Total  workload  for  the  operator  after  beginning  the  next  task. 

S  -  Operator’s  workload  threshold  value. 

An  example  of  a  strategy  a  designer  might  create  using  these  variables  is  “if  P  >  H, 
then  D,  else  C.”  In  this  example,  D  and  C  are  the  pre-defined  strategies  C  and  D 
that  are  embedded  in  the  software.  This  system  designer-created  strategy 
translates  to  “if  the  priority  of  the  new  task  is  higher  than  that  of  any  other 
ongoing  task,  then  perform  the  new  task  and  suspend  the  ongoing  tasks  until 
workload  goes  below  the  threshold.”  The  critical  aspect  of  the  workload 
management  strategies  created  by  the  system  designer,  as  well  as  the  pre-defined 
workload  management  strategies,  is  that  they  represent  dynamic  task  scheduling. 
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based  on  operator  workload.  In  other  words,  the  system  designer  does  not 
specify  exactly  which  task  the  operator  will  do  next  in  performing  the  mission. 
The  next  task(s)  will  be  determined  in  part,  based  on  the  momentary  workload  of 
the  operator  (Archer  &  Lockett,  1997).  The  designers  can  view  the  effect  of  the 
workload  management  strategies  and  the  interaction  of  the  workload  and 
performance  for  the  mission  by  viewing  the  advanced  workload  analysis  output 
reports. 

The  advanced  workload  analysis  reports  include  a  mission  summary,  critical  path 
report,  workload  graphs,  and  reports  describing  the  operator  activity,  overload, 
and  channel  conflicts.  From  these  reports,  system  designers  can  view  the  total 
workload  value  over  time  for  each  operator  of  the  system.  Because  this  value  is 
on  an  ordinal  scale,  it  allows  the  system  designer  to  make  only  relative 
comparisons  of  workload  at  different  times  during  the  mission.  This  means  that 
the  system  designer  should  not  compare  specific  overall  workload  numbers. 
Instead,  the  designer  examines  the  workload  graphs  and  determines  where  the 
workload  peaks  are  and  which  tasks  were  operating  at  that  time  and  contributed  to 
the  peaks.  The  designer  can  then  select  Aese  tasks  as  candidates  for  redesign, 
automation,  or  reallocation  to  another  crew  member  (Archer,  1998).  Furthermore, 
the  process  of  building  and  inputting  data  into  the  models  helps  the  system 
designers  to  think  about  those  interfaces  and  tasks  that  are  contributing  factors  to 
worUoad  and  performance. 


8.  Validation 


The  IMPRINT  workload  analysis  options  help  the  system  designer  to  make 
accurate  predictions  about  operator  workload  and  its  effect  on  performance  of  a 
proposed  system.  In  order  for  the  analyst  to  make  the  correct  design  decisions 
using  the  IMPRINT  workload  options,  however,  the  predicted  mental  workload  and 
performance  estimates  obtained  from  the  IMPRINT  model  must  be  accurate 
representations  of  actual  mental  workload  and  performance  in  the  real  world. 
Therefore,  to  evaluate  the  validity  of  the  IMPRINT  workload  predictions, 
researchers  at  ARL  participated  in  an  international  effort  to  determine  which 
workload  models  make  the  best  predictions  of  workload  and  performance  and  during 
what  conditions  these  capabilities  exist.  Workload  models  from  the  United 
Kingdom  (POP),  Canada  (IP/PCT),  and  the  United  States  (IMPRINT  advanced 
workload  option)  were  evaluated  with  the  same  set  of  data  collected  during  two 
experiments  at  Wright-Patterson  Air  Force  Base.  In  fulfillment  of  its  part  of  this 
effort,  ARL  built  IMPRINT  advanced  workload  models  to  represent  the  two 
experiments.  Preliminary  data  analyses  comparing  the  experimental  workload 
ratings  with  the  IMPRINT  predicted  workload  ratings  indicate  that  IMPRINT 
advanced  workload  method  has  good  predictive  validity.  Specifically,  the  workload 
ratings  from  the  IMPRINT  models  correlated  with  the  subjective  workload  ratings 
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given  by  the  participants  in  the  Wright-Patterson  experiment.  Furthermore,  the 
differences  between  experimental  conditions  predicted  by  the  IMPRINT  models 
were  the  same  as  the  differences  between  experimental  conditions  provided  by  the 
performance  measures  gathered  during  the  experiment  (Mitchell,  1999).  This 
validation  effort  proved  that  the  IMPRINT  advanced  workload  analysis  option  can 
be  used  to  provide  valid  predictions  of  the  mental  workload  of  individual  operators. 
Indeed,  the  IMPRINT  workload  options  have  been  used  to  make  valid  predictions 
about  the  mental  workload  of  the  operators  of  military  systems. 


9.  Applications 


System  designers  have  used  the  VACP  workload  option  for  a  number  of  military 
applications.  They  used  MAN-SEVAL,  which  is  synonymous  with  the  VACP 
workload  option  in  IMPRINT,  to  model  the  U.S.  Army’s  Land  Warrior  system. 
The  designers  built  baseline  models  of  a  squad  leader  and  a  squad  member 
equipped  with  current  equipment.  They  then  built  models  of  the  squad  member 
and  squad  leader  wearing  Land  Warrior  equipment.  The  designers  compared  the 
two  model  sets  to  determine  the  effect  of  the  Land  Warrior  equipment  on  mission 
performance  (Adkins,  Murphy,  Hemenway,  Archer,  &  Bayless,  1996). 

Designers  have  also  used  the  IMPRINT  VACP  option  to  model  joint  base  station 
variant  (JBS  VI)  missions.  IMPRINT  analyses  were  conducted  to  assess  skill 
and  requirements,  workload,  and  mission  performance  to  assist  in  determining  the 
appropriate  quantity  and  military  occupational  specialty  of  JBS  (VI)  operators 
(Malkin,  Allender,  Kelley,  O’Brien,  &  Graybill,  1997)^  Most  recently,  system 
designers  have  used  IMPRINT  to  build  models  of  the  Crusader  155-mm  howitzer 
system  to  evaluate  the  effects  of  crew  size,  continuous  operations,  and  number  of 
vehicle  work  stations  on  mission  performance  (Beideman,  Munro,  &  Allender,  in 
press). 

WinCrew,  the  stand-alone  version  of  the  advanced  workload  analysis,  has  been 
used  to  model  the  workload  and  functional  allocations  of  the  bridge  activities 
aboard  a  U.S.  Navy  destroyer  (Archer,  Lewis,  &  Lockett,  1996).  During  this 
study,  system  designers  built  a  baseline  model  of  the  existing  nine-person  bridge 
crew.  Then  they  built  three  more  models  to  represent  different  configurations  of 
automation  and  function  and  task  allocations  for  a  reduced  crew  of  three  (Archer 
&  Lockett,  1997).  The  U.S.  Federal  Aviation  Association  also  used  WinCrew  to 
evaluate  alternate  crew  and  function  allocations  for  oceanic  air  traffic  controllers 
(Archer  &  Lockett,  1997). 


19 


10.  Future  Needs 


In  combat  situations,  military  system  operators  usually  work  in  crews  or  as  teams 
rather  than  in  isolation.  Therefore,  ARL  researchers  plan  to  incorporate  the 
prediction  of  team  mental  workload  and  performance  into  the  IMPRINT 
workload  options.  Recent  research  has  demonstrated  that  team  performance  is 
more  than  just  the  sum  of  the  performances  of  individuals  (Bowers,  Brairn,  & 
Morgan,  in  press).  Therefore,  system  designers  need  guidelines  for  team 
performance  in  the  models  of  the  proposed  system.  Currently,  the  IMPRINT 
software  provides  the  designer  with  some  capabilities  for  modeling  the  operator  as 
part  of  a  crew.  For  example,  one  of  the  workload  management  strategies 
employed  by  a  crew  member  is  to  have  a  contingency  operator  perform  the  new 
task  in  an  overload  situation.  Therefore,  the  designer  can  assign  one  crew  member 
as  the  primary  person  or  operator  who  will  perform  a  task  and  another  individual 
as  a  “contingency  operator.”  The  contingency  operator  is  someone  who  would 
not  normally  perform  that  task  but  has  access  to  the  controls  and  displays  to  do 
it.  If  the  primary  operator  becomes  overloaded,  then  the  contingency  operator 
may  be  able  to  perform  the  task  for  him  or  her.  If  performing  this  additional  task 
will  overload  the  contingency  operator,  then  the  task  goes  back  to  the  primary 
operator. 

Although  IMPRINT  features  provide  some  preliminary  capability  for  the  analyst 
to  handle  team  performance,  they  do  not  provide  the  analyst  with  any  theoretical 
basis  for  how  teams  handle  multiple  tasks.  Therefore,  the  goal  of  the  present 
ARL  advanced  workload  IMPRINT  research  effort  is  to  review  the  literature 
about  team  performance,  especially  as  it  relates  to  mental  workload,  and  to 
identify  data  sources  and  data  that  can  be  used  to  further  develop  the  IMPRINT 
advanced  workload  team  performance  modeling  capabilities. 
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